Preferred modalities in dialogue systems
نویسندگان
چکیده
This research describes which modalities are preferred in particular contexts when interacting with a multi-modal dialogue system. The trade-off between three factors is investigated: (i) speech recognition performance, (ii) efficiency of input modality and (iii) the system’s output modality. Four versions were developed of a multimodal examinator to be used in elementary school. The versions differed in recognition performance (‘perfect’ vs. realistic) and output modality (speech or text). In all systems, subjects could provide input via speaking or typing. Answer length in characters was used as a measure of efficiency. Results show that both speech recognition performance and efficiency have a strong impact on preferred modalities. No effect was found of the system’s output modality.
منابع مشابه
Using speech and gesture to explore user states in multimodal dialogue systems
Modern dialogue systems should interpret the users’ behavior and mind in the same way as human beings do. That means in a multimodal manner, where communication is not limited to verbal utterances, as is the case for most state-of-the-art dialogue systems, several modalities are involved, e.g., speech, gesture, and facial expression. The design of a dialogue system must adapt its concept to mul...
متن کاملModeling Lateral Communication in Holonic Multi Agent Systems
Agents, in a multi agent system, communicate with each other through the process of exchanging messages which is called dialogue. Multi agent organization is generally used to optimize agents’ communications. Holonic organization demonstrates a self-similar recursive and hierarchical structure in which each holon may include some other holons. In a holonic system, lateral communication occurs b...
متن کاملAn Adaptive Approach to Collecting Multimodal Input
Multimodal dialogue systems allow users to input information in multiple modalities. These systems can handle simultaneous or sequential composite multimodal input. Different coordination schemes require such systems to capture, collect and integrate user input in different modalities, and then respond to a joint interpretation. We performed a study to understand the variability of input in mul...
متن کاملDialogue Systems Go Multimodal: The SmartKom Experience
Multimodal dialogue systems exploit one of the major characteristics of humanhuman interaction: the coordinated use of different modalities. Allowing all of the modalities to refer to and depend upon each other is a key to the richness of multimodal communication. We introduce the notion of symmetric multimodality for dialogue systems in which all input modes (e.g., speech, gesture, facial expr...
متن کاملMultimodal Dialogue System Grammars∗
We describe how multimodal grammars for dialogue systems can be written using the Grammatical Framework (GF) formalism. A proof-of-concept dialogue system constructed using these techniques is also presented. The software engineering problem of keeping grammars for different languages, modalities and systems (such as speech recognizers and parsers) in sync is reduced by the formal relationship ...
متن کامل